How isotropic kernels perform on simple invariants

Abstract We investigate how the training curve of isotropic kernel methods depends on symmetry task to be learned, in several settings. (i) consider a regression task, where target function is Gaussian random field that only d ∥ variables, fewer than input dimension d . compute expected test error ϵ follows $\epsilon\sim p^{-\beta}$?> ϵ ∼ p − β p size set. find β ∼ 1/ independently , supporting previous findings presence invariants does not resolve curse dimensionality for regression. (ii) Next we support-vector binary classification and introduce stripe model data label single coordinate $y(\underline x) = y(x_1)$?> y ( x accent="true" stretchy="true">_ stretchy="false">) = 1 corresponding parallel decision boundaries separating labels different signs, there no margin at these interfaces. argue confirm numerically that, large bandwidth, $\beta \frac{d-1+\xi}{3d-3+\xi}$?> + ξ 3 ξ ∈ (0, 2) exponent characterizing singularity origin. This estimation improves classical bounds obtainable from Rademacher complexity. In this setting since $\beta\rightarrow 1/3$?> stretchy="false">→ / as $d\rightarrow\infty$?> ∞ (iii) spherical which y(||\underline x||)$?> | (iv) model, show if are compressed along their by some factor λ (an operation believed take place deep networks), reduced $\lambda^{-\frac{2(d-1)}{3d-3+\xi}}$?> λ 2